Article 5319

Title of the article

USING REINFORCEMENT LEARNING IN CREATING SELF-ADAPTIVE SOFTWARE 

Authors

Bozhday Aleksandr Sergeevich, Doctor of engineering sciences, professor, sub-department of CAD systems, Penza State University (40 Krasnaya street, Penza, Russia), E-mail: bozhday@yandex.ru
Evseeva Yuliya Igorevna, Candidate of engineering sciences, associate professor, sub-department of CAD systems, Penza State University (40 Krasnaya street, Penza, Russia), E-mail: shymoda@mail.ru
Artamonov Dmitriy Vladimirovich, Doctor of engineering sciences, professor, First Vice Rector of Penza State University (40 Krasnaya street, Penza, Russia), E-mail: aius@pnzgu.ru 

Index UDK

004.4 

DOI

10.21685/2072-3059-2019-3-5 

Abstract

Background. Creating effective software self-adaptation techniques is a fairly discussed topic in the software development community. The urgency of the problem is due to the high complexity of modern software systems, the significant costs of their development and maintenance. The problem of the high complexity of the system also raises the problem of the quality of its functioning: in the case of software systems having a complex structure and behavior, it is rather difficult to take into account the whole spectrum of situations in which they will exhibit undesirable behavior for users. A similar problem is associated with the optimal functioning of such systems: the software must independently determine situations in which it is necessary to increase or decrease the amount of resources consumed in order to optimize performance. Thus, the aim of the work is to create a new universal technique for self-adaptation of software systems that can solve the problem of improving the quality of software functioning.
Materials and methods. As the main method for solving the problem, a modified method of training with reinforcement was used. The difference from the classical method is that the procedure underlying it is model-oriented (it is based on the domain model in which the system operates) and takes into account the factor of uncertainties arising during the operation of the software.
Results. The main results of the work include: a review and classification of existing methods for implementing self-adaptive behavior of software systems; a modified learning procedure with reinforcement, which operates on the basis of the model of the subject area of the system and takes into account the factor of arising uncertainties.
Conclusions. The proposed reinforcement learning procedure is applicable to a wide range of software systems. Its application will solve the problems associated with the behavior of complex software systems: the optimality of resource consumption in various situations, the uncertainties that arise during the operation of the program. 

Key words

machine learning, reinforcement learning, self-adaptive software 

Download PDF
References

1. Ghezzi C., Salvaneschi G., Pradella M. Science of Computer Programming. Amsterdam: Elsevier, 2015, pp. 20–43.
2. Lei Y., Ben K., He Z. 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). New Jersey: IEEE, 2015, pp. 255–265.
3. Li Y., Li L., Wang W., Wu T. IEEE Conference on Open Systems (ICOS). New Jersey: IEEE, 2017, pp. 145–167.
4. Bencomo N., Belaggoun A. International Working Conference on Requirements Engineering: Foundation for Software Quality. San Francisco: IEEE, 2013, pp. 221–237.
5. Belhaj N., Belaid D., Mukhtar H. 2018 IEEE International Conference on Services Computing (SCC). San Francisco: IEEE, 2018, pp. 139–156.
6. Dongsun K., Sooyong P. 2009 ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems. San Francisco: IEEE, 2009, pp. 178–184.
7. Colette R., Carine S., Camille B. IEEE Transactions on Software Engineering. San Francisco: IEEE, 1998, pp. 114–127.
8. Naeem E., Sam M. Uncertainty inSelf-Adaptive Software Systems. Berlin: Springer, 2013, pp. 214–238.
9. Villegas N. M., Muller H. A. The Smart Internet. Berlin: Springer, 2010, pp. 289–318.
10. Han H. International Conference on Ubiquitous Information Management and Communication. New Jersey: IEEE, 2015, pp. 156–178.
11. Bozhday A. S., Evseeva Yu. I. Izvestiya vysshikh uchebnykh zavedeniy. Povolzhskiy region. Tekhnicheskie nauki [University proceedings. Volga region. Engineering sciences]. 2018, no. 2 (46), pp. 74–86. [In Russian]

 

Дата создания: 14.01.2020 10:11
Дата обновления: 22.01.2020 13:36